An invariance property of kernel based predictors

We consider kernel based learning methods for regression and analyze what happens to the risk minimizer when new variables, statistically independent of input and target variables, are added to the set of input variables. We find that the risk minimizer remains unchanged if we constrain the risk minimization to hypothesis spaces induced by suitable kernel functions. We show that not all kernel induced hypothesis spaces enjoy this property. We present sufficient conditions ensuring that risk minimizer does not change, and show that they hold for polynomial and Gaussian RBF kernels. We also provide examples of kernel induced hypothesis spaces whose risk minimizer changes if independent variables are added as input. Institute of Intelligent Systems for Automation C.N.R. Via Amendola 122/D-I 70126 Bari Italy. TIRES-Center of Innovative Technologies for Signal Detection and Processing, University of Bari, Italy. Dipartimento Interateneo di Fisica, University of Bari, Italy. Istituto Nazionale di Fisica Nucleare, Sezione di Bari, Italy.